test: Add --ignore and --collect-only and use them in CI - #1683
Merged
Conversation
Contributor
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1683 +/- ##
=======================================
Coverage 97.93% 97.93%
=======================================
Files 178 179 +1
Lines 15995 16063 +68
Branches 3664 3679 +15
=======================================
+ Hits 15664 15732 +68
Misses 252 252
Partials 79 79
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
chfast
commented
Aug 29, 2026
|
|
||
| bool collect_only = false; | ||
| app.add_flag("--collect-only", collect_only, | ||
| "List the id of each test that would run, one per line, and exit."); |
| #include <test/utils/test_files.hpp> | ||
| #include <test/utils/test_report.hpp> | ||
| #include <iostream> | ||
| #include <span> |
| #include <test/utils/test_files.hpp> | ||
| #include <test/utils/test_report.hpp> | ||
| #include <iostream> | ||
| #include <span> |
There was a problem hiding this comment.
Pull request overview
Adds pytest-style test collection controls and migrates CI away from GoogleTest filtering.
Changes:
- Adds repeatable
--ignoreand--collect-onlyoptions. - Adds path-filtering unit and integration tests.
- Converts CI exclusions and collection listings to the new options.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
test/utils/test_files.hpp |
Declares ignore filtering API. |
test/utils/test_files.cpp |
Implements component-aware filtering. |
test/unittests/test_files_test.cpp |
Tests ignore behavior. |
test/unittests/CMakeLists.txt |
Registers the new unit test. |
test/statetest/statetest.cpp |
Adds collection options to state tests. |
test/blockchaintest/blockchaintest.cpp |
Adds collection options to blockchain tests. |
test/integration/statetest/CMakeLists.txt |
Tests state-test collection behavior. |
test/integration/blockchaintest/CMakeLists.txt |
Tests blockchain-test collection behavior. |
test/integration/export/CMakeLists.txt |
Migrates exported-test exclusion. |
circle.yml |
Migrates CI filters to ignored paths. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+42
to
+46
| auto p = prefix.lexically_normal(); | ||
| if (p.filename().empty()) | ||
| p = p.parent_path(); | ||
| // An empty prefix, which an unset variable expands to, names nothing rather than everything. | ||
| return !p.empty() && std::ranges::mismatch(p, path).in1 == p.end(); |
chfast
force-pushed
the
test/collect-only-ignore
branch
2 times, most recently
from
August 29, 2026 20:18
c9fbaa0 to
8737a9e
Compare
Both flags follow pytest. --ignore drops a path, relative to a test directory, from the collection; it is repeatable and matches whole path components, so --ignore bc4895 keeps bc4895-withdrawals. --collect-only prints the id of each test that would run, one per line: the file for a test collected from a directory, path::case for a file given directly. The gtest flags keep working and nothing passes the new ones yet.
Every path CI excluded through --gtest_filter is a plain path, so the patterns port one for one; the run_execution_spec_tests filter parameter becomes the flags themselves. --collect-only replaces --gtest_list_tests in the three listing tests, which now pin the collection order across roots instead of gtest's regrouping by suite name.
The five excluded names are legacy GeneralStateTests suites which CI has not run since it moved to evmone-blockchaintest, and the exclusion was inert anyway: the execution-spec-tests jobs passed --gtest_filter='*', which overrode it.
chfast
force-pushed
the
test/collect-only-ignore
branch
from
August 29, 2026 20:42
8737a9e to
df54d9f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both flags follow pytest.
--ignoredrops a path from the collection; it is repeatable and matcheswhole path components, so
--ignore bc4895keepsbc4895-withdrawals. The path is relative to thetest directory being searched rather than to the working directory as pytest resolves it, which is
what keeps the CI exclusions below one word each.
--collect-onlyprints the path of each collectedtest, one per line: the file for a test collected from a directory,
path::casefor a file givendirectly.
Every path CI excluded through
--gtest_filteris a plain path, so the patterns port one for one,and
--collect-onlyreplaces--gtest_list_testsin the three listing tests, which now pin thecollection order across roots instead of gtest's regrouping by suite name.
ethereum-testsruns thesame 479 ValidBlocks and 221 InvalidBlocks tests as on master, and the execution-spec-tests jobs the
same 8281 state and 8731 blockchain tests.
The last commit drops statetest's built-in skip list: those five names are legacy GeneralStateTests
suites CI has not run since it moved to evmone-blockchaintest, and the list was inert anyway because
the jobs passed
--gtest_filter='*'over it. After this nothing in the repo passes a--gtest_*flag except
evmone-unittests, which keeps gtest.